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AMENDMENTS TO THE CLAIMS 
Please amend Claims 18, 34 and 57 as indicated. 

1-17. (Canceled) 

18. (Currently amended) A method of testing a speech recognizer, the method 
comprising: 

receiving a plurality of digital audio data files, each audio file comprising audio 
recorded in response to a first prompt by a speech recognition application; 

receiving a grammar associated with the first prompt, the grammar comprising a 
plurality of concepts, each concept having a set of phrases organized under a single idea, the 
idea representing an expected response to the first prompt; 

producing a first d e cod e recognition r esult for each audio data file based at least in part 
on the grammar using the speech recognizer ; 

receiving a user-defined t ranscript of each audio filet and scoring the first decode 
recognition results for each audio data file based at least in part on the transcript of each audio 
file; 

modifying the grammar based on the scoring of the first recognition result of each 
audio data file; 

producing a second recognition result for each audio data file based on the modified 
grammar using the speech recognizer; 

using the user-defined transcript of each audio data file to score the second recognition 
result for each audio data file; 

comparing the scoring of the first recognition result with the second result for each 
audio data file: and 

outputting the first or second recognition result for each audio data file based on the 
comparison. 

19. (Canceled) 

20. (Previously presented) The method of Claim 18, wherein the first decode result 
comprises concepts, phrases, words, and/or phonemes. 

21. (Previously presented) The method of Claim 18, wherein the first decode result 
comprises a confidence score. 
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22. (Original) The method of Claim 18, further comprising displaying a result of the 
scoring. 

23. (Original) The method of Claim 18, further comprising creating and/or modifying a 
response file associated with the audio data file. 

24. (Original) The method of Claim 18, wherein the response file comprises the audio 
file, a portion of the grammar associated with the audio file, the decode result, and/or the 
transcript. 

25-33. (Canceled) 

34. (Currently amended) A system for testing a speech recognizer, the system comprising: 
an audio recorder module for receiving a plurality of digital audio data files, each data 

file comprising audio recorded in response to a first prompt of a speech recognition 
application; 

a grammar editor module configured to access and modify a grammar based on scoring 
of a recognition result, the grammar comprising a plurality of concepts, each concept having a 
set of phrases organized under a single idea, the idea representing an expected response to the 
first prompt; 

a speech recognition engine configured to output a first recognition result for each 
audio data file of the plurality of digital audio data files and the accessed grammar using the 
speech recognizer ; and 

a scoring module configured to score the first recognition results based at least in part 
on a-user-defined transcript of each audio data file of the plurality of audio data files, 
wherein said speech recognition engine is configured to output a second recognition result for 
each audio data file of the plurality of digital audio data files based on said modified grammar 
and wherein said scoring module is further configured to compare the scoring of the first 
recognition result with the second recognition result for each audio data file and output the 
first or second recognition result for each audio data file based on said comparison. 

35. (Original) The system of Claim 34, further comprising a user interface. 

36. (Original) The system of Claim 34, wherein the user interface comprises a graphical 
user interface. 
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37. (Original) The system of Claim 36, wherein the graphical user interface is configured 
to display an output of the scoring module. 

38. (Original) The system of Claim 34, wherein the recognition result comprises a 
confidence score. 

39. (Original) The system of Claim 34, wherein the recognition result comprises a 
concept, phrase, word, or phoneme. 

40. (Previously presented) The system of Claim 34, wherein the recognition result 
comprises an indication of an acoustic model used by the speech recognizer in decoding each 
audio data file. 

41. (Original) The system of Claim 40, wherein the recognition result comprises an 
acoustic model score. 

42. (Previously presented) The system of Claim 34, further comprising a response file for 
logically associating an audio data file, the transcript, the recognition result, and/or an output 
of the scoring module. 

43-50. (Canceled) 

51. (Previously presented) The system of Claim 34, wherein the speech recognition 
engine is configured to transmit the recognition result to a tuner module for processing. 

52. (Previously presented) The system of Claim 51, further comprising a tuner module 
configured to transmit digital audio input to the audio recorder module and grammar to the 
grammar editor module. 

53. (Previously presented) The system of Claim 34, further comprising a test module 
configured to initiate a testing cycle by processing and transmitting digital audio input and 
grammar to the speech recognition engine. 

54. (Previously presented) The system of Claim 53, wherein the speech recognition 
engine is configured to transmit the recognition result to a tuner module for processing. 

55. (Previously presented) The system of Claim 54, further comprising a tuner module 
configured to transmit digital audio data and grammar to the test module. 

56. (Previously presented) The system of Claim 36, wherein the graphical user interface is 
configured to display the digital audio input and the accessed grammar. 

57. (Currently amended) A system for testing a speech recognizer, the system comprising: 
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an audio data input module for receiving a plurality of digital audio data files, each 
audio data file comprising audio recorded in response to a first prompt from a speech 
recognition application; 

a grammar editor module configured to access and modify a grammar, the grammar 
comprising a plurality of concepts, each concept having a set of phrases organized under a 
single idea, the idea representing an expected response to the first prompt; 

a test module configured to initiate a first testing cycle, the testing cycle comprising 
transmitting the plurality of digital audio data files and the grammar to a speech recognition 
engine; and 

a scoring module configured to receive a first recognition result for each of the 
plurality of audio data files from the speech recognition engine, and further configured to 
score the first recognition results based at least in part on a user-defined transcript of the audio 
input, 

wherein said test module is configured to initiate a second testing cycle comprising 
transmitting the plurality of digital audio data files and the modified grammar to said speech 
recognition engine, and wherein said scoring module is configured to receive a recognition 
result for said second testing cycle for each of the plurality of audio data files based on the 
modified grammar, from the speech recognition engine and is further configured to compare 
the scoring of the first recognition result with the second recognition result for each audio data 
file and output the first or second recognition result for each audio data file based on said 
comparison. 

58. (Previously presented) The system of Claim 57, further comprising a speech 
recognition engine configured to output a recognition result to the scoring module for each of 
the plurality of audio data files received from the test module. 

59. (Previously presented) The system of Claim 58, wherein the speech recognition 
engine is further configured to transmit the recognition results to a tuner module for 
processing. 

60. (Previously presented) The system of Claim 59, further comprising a tuner module 
configured to transmit digital audio data and grammar to the test module. 

61. (Previously presented) The system of Claim 58, further comprising a user interface. 



-5- 



Appl. No. 
Filed 



10/725,281 
December 1, 2003 



62. (Previously presented) The system of Claim 61, wherein the user interface comprises 
a graphical user interface. 

63. (Previously presented) The system of Claim 62, wherein the graphical user interface is 
configured to display an output from a the scoring modulev 

64. (Previously presented) The system of Claim 62, wherein the graphical user interface is 
configured to display a digital audio data file and the accessed grammar. 

65. (Previously presented) The system of Claim 58, wherein the recognition result 
comprises a confidence score. 

66. (Previously presented) The system of Claim 58, wherein the recognition result 
comprises a concept, phrase, word, or phoneme. 

67. (Previously presented) The system of Claim 58, wherein the recognition result 
comprises an indication of an acoustic model used by the speech recognition engine in 
decoding an audio data file. 

68. (Previously presented) The system of Claim 66, wherein the recognition result 
comprises an acoustic model score. 

69. (Previously presented) The system of Claim 58, further comprising a response file for 
logically associating an audio data file, the transcript, the recognition result, and/or an output 
of the module configured to output a recognition result. 

70. (Previously presented) The method of Claim 22, further comprising displaying a result 
of the scoring on a user interface. 

71. (Previously presented) The method of Claim 70, wherein the user interface is a 
graphical user interface 

72. (Previously presented) The method of Claim 18, further comprising transmitting the 
first decode results to a tuner module for processing. 

73. (Previously presented) The method of Claim 18, further comprising: 
modifying the grammar; 

producing a second decode result of each digital audio file based at least in part on the 
modified grammar; and 

scoring the second decode results based at least in part on the transcript of each audio 

file. 
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74. (Previously presented) The method of Claim 73, further comprising comparing the 
scoring of the first decode results and the scoring of the second decode results. 

75. (Previously presented) The method of Claim 18, wherein each of the set of phrases 
comprises a word, a word block, a BNF construct, or a phoneme block. 

76. (Previously presented) The method of Claim 18, further comprising: 

receiving a second plurality of digital audio data files, each audio file comprising 
audio recorded in response to a second prompt by the speech recognition application; 

receiving a second grammar associated with the second prompt, wherein the second 
grammar comprises a plurality of concepts, each concept having a set of phrases organized 
under a single idea, the idea representing an expected response to the second prompt; 

producing a second decode result for each audio file in the second plurality of digital 
audio data files based at least in part on the second grammar; 

receiving a transcript of each audio file in the second plurality of audio data files; and 
scoring the second decode results based at least in part on the transcripts of each of the second 
plurality of digital audio files. 

77. (Previously presented) The method of Claim 18, wherein scoring the decode results 
comprises generating statistics on the accuracy of the decode results with respect to each 
transcript, the statistics comprising word error rate, concept error rate, and average confidence 
scores for correct and incorrect results. 

78. (Previously presented) The system of Claim 34, wherein the system is configured to, 
iteratively, modify the grammar based on a previous scoring of recognition results using the 
grammar editor module, output a recognition result for each audio data file based on the 
modified grammar using the speech recognition engine, and use the user-defined transcript of 
each audio data file to score the modified grammar recognition results using the scoring 
module. 

79. (Previously presented) The system of Claim 57, wherein the grammar editor module 
is further configured to modify the grammar based on the scoring of the recognition results, 
the test module is further configured to transmit the plurality of audio data files and the 
modified grammar to a speech recognition engine, and the scoring module is further 
configured to receive a recognition result based on the modified grammar from the speech 
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recognition engine for each of the plurality of audio data files and to score the recognition 
results based at least in part on the user-defined transcript. 



